Times are changing, bias isn't: A meta-meta-analysis on publication bias detection practices, prevalence rates, and predictors in industrial/organizational psychology.

Effect misestimations plague Psychological Science, but advances in the identification of dissemination biases in general and publication bias in particular have helped in dealing with biased effects in the literature. However, the application of publication bias detection methods appears to be not equally prevalent across subdisciplines. It has been suggested that particularly in I/O Psychology, appropriate publication bias detection methods are underused. In this meta-meta-analysis, we present prevalence estimates, predictors, and time trends of publication bias in 128 meta-analyses that were published in the Journal of Applied Psychology (7,263 effect sizes, 3,000,000 + participants). Moreover, we reanalyzed data of 87 meta-analyses and applied nine standard and more modern publication bias detection methods. We show that (a) the bias detection method applications are underused (only 41% of meta-analyses use at least one method) but have increased in recent years, (b) those meta-analyses that apply such methods now use more, but mostly inappropriate methods, and (c) the prevalence of potential publication bias is concerning but mostly remains undetected. Although our results indicate somewhat of a trend toward higher bias awareness, they substantiate concerns about potential publication bias in I/O Psychology, warranting increased researcher awareness about appropriate and state-of-the-art bias detection and triangulation. Embracing open science practices such as data sharing or study preregistration is needed to raise reproducibility and ultimately strengthen Psychological Science in general and I/O Psychology in particular. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

After all, it is an outdoor sport: Meta-analytic evidence for negative associations between wind compensation points and round scores in ski jumping competitions.

Wind conditions are well-known to affect results of ski jumping competitions. To alleviate effects of different head or tail winds and differences in inrun length due to jury or coaches' decisions, the FIS (Féderation Internationale de Ski) has adopted a wind and gate compensation system since January 2010. However, the accuracy and fairness of the resulting compensation points are often questioned by athletes, spectators, and media commentators alike but have not yet been thoroughly investigated. In the present meta-analysis, we present evidence for systematic negative associations of wind points but positive associations of gate points with round scores across all World Cup and World Championship competitions of men in the ski jumping seasons 2016/2017 and 2017/2018 (k = 35 and 30, respectively). Moreover, our results indicate that absolute correlations between wind points and round scores increased in presence of larger wind point variability, thus indicating lower accuracy of compensation systems when conditions are more variable. Additionally, there was a trend for larger wind point malcompensations on larger jumping hills. Our results suggest that the current wind point compensation formula as well as gate factors should be reevaluated to prevent systematically biased point awards within and across competitions.

Effect Declines Are Systematic, Strong, and Ubiquitous: A Meta-Meta-Analysis of the Decline Effect in Intelligence Research.

Empirical sciences in general and psychological science in particular are plagued by replicability problems and biased published effect sizes. Although dissemination bias-related phenomena such as publication bias, time-lag bias, or visibility bias are well-known and have been intensively studied, another variant of effect distorting mechanisms, so-called decline effects, have not. Conceptually, decline effects are rooted in low initial (exploratory) study power due to strategic researcher behavior and can be expected to yield overproportional effect declines. Although decline effects have been documented in individual meta-analytic investigations, systematic evidence for decline effects in the psychological literature remains to date unavailable. Therefore, we present in this meta-meta-analysis a systematic investigation of the decline effect in intelligence research. In all, data from 22 meta-analyses comprising 36 meta-analytical and 1,391 primary effect sizes (N = 697,000+) that have been published in the journal Intelligence were included in our analyses. Two different analytic approaches showed consistent evidence for a higher prevalence of cross-temporal effect declines compared to effect increases, yielding a ratio of about 2:1. Moreover, effect declines were considerably stronger when referenced to the initial primary study within a meta-analysis, yielding about twice the magnitude of effect increases. Effect misestimations were more substantial when initial studies had smaller sample sizes and reported larger effects, thus indicating suboptimal initial study power as the main driver of effect misestimations in initial studies. Post hoc study power comparisons of initial versus subsequent studies were consistent with this interpretation, showing substantially lower initial study power of declining, than of increasing effects. Our findings add another facet to the ever accumulating evidence about non-trivial effect misestimations in the scientific literature. We therefore stress the necessity for more rigorous protocols when it comes to designing and conducting primary research as well as reporting findings in exploratory and replication studies. Increasing transparency in scientific processes such as data sharing, (exploratory) study preregistration, but also self- (or independent) replication preceding the publication of exploratory findings may be suitable approaches to strengthen the credibility of empirical research in general and psychological science in particular.